CenterTransFuser: radar point cloud and visual information fusion for 3D object detection

Abstract Sensor fusion is an important component of the perception system in autonomous driving, and radar point cloud information camera visual can improve capability vehicles. However, most existing studies ignore extraction local neighborhood only consider shallow between two modalities based on extracted global information, which cannot perform a deep cross-modal contextual interaction. Meanwhile, data preprocessing, noise usually filtered by depth derived from image feature prediction, such methods affect accuracy branching to generate regions interest effectively filter out irrelevant points. This paper proposes CenterTransFuser model that makes full use millimeter-wave enable heterogeneous information. Specifically, new interaction called cross-transformer explored, cooperatively exploits cross-multiple attention joint mine complementary adaptive thresholding filtering method designed reduce modality-independent projected onto image. The evaluated challenging nuScenes dataset, it achieves excellent performance. Particularly, detection significantly improved for pedestrians, motorcycles, bicycles, showing superiority effectiveness proposed model.